[OptBisect] Add support for selecting ranges of passes and refactor DebugCounter to use a shared Range API. #152393

YonahGoldberg · 2025-08-06T21:44:41Z

Add -opt-bisect-skip=<idx> option for skipping pass at index <idx> in the pipeline.

OptBisect allows you to binary search for a pass causing some error. It is useful as a next step to disable that pass as a sanity check and to see if the issue is isolated to one pass.

YonahGoldberg · 2025-08-06T21:45:11Z

@AlexMaclean @nikic

llvmbot · 2025-08-06T21:45:17Z

@llvm/pr-subscribers-llvm-support

@llvm/pr-subscribers-llvm-ir

Author: Yonah Goldberg (YonahGoldberg)

Changes

Add -opt-bisect-skip=<idx> option for skipping pass at index <idx> in the pipeline.

OptBisect allows you to binary search for a pass causing some error. It is useful as a next step to disable that pass as a sanity check and to see if the issue is isolated to one pass.

Full diff: https://github.com/llvm/llvm-project/pull/152393.diff

3 Files Affected:

(modified) llvm/include/llvm/IR/OptBisect.h (+9-1)
(modified) llvm/lib/IR/OptBisect.cpp (+8-1)
(added) llvm/test/Other/opt-bisect-skip.ll (+15)

diff --git a/llvm/include/llvm/IR/OptBisect.h b/llvm/include/llvm/IR/OptBisect.h
index d813ae933d65e..9dcc0077552d4 100644
--- a/llvm/include/llvm/IR/OptBisect.h
+++ b/llvm/include/llvm/IR/OptBisect.h
@@ -14,6 +14,7 @@
 #ifndef LLVM_IR_OPTBISECT_H
 #define LLVM_IR_OPTBISECT_H
 
+#include "llvm/ADT/SmallSet.h"
 #include "llvm/ADT/StringRef.h"
 #include "llvm/ADT/StringSet.h"
 #include "llvm/Support/Compiler.h"
@@ -67,7 +68,13 @@ class LLVM_ABI OptBisect : public OptPassGate {
                      StringRef IRDescription) const override;
 
   /// isEnabled() should return true before calling shouldRunPass().
-  bool isEnabled() const override { return BisectLimit != Disabled; }
+  bool isEnabled() const override {
+    return BisectLimit != Disabled || !BisectSkipNumbers.empty();
+  }
+
+  /// Add pass at index SkipNumber to the list of passes to skip
+  /// during bisection.
+  void addSkip(int SkipNumber) { BisectSkipNumbers.insert(SkipNumber); }
 
   /// Set the new optimization limit and reset the counter. Passing
   /// OptBisect::Disabled disables the limiting.
@@ -81,6 +88,7 @@ class LLVM_ABI OptBisect : public OptPassGate {
 private:
   int BisectLimit = Disabled;
   mutable int LastBisectNum = 0;
+  SmallSet<int, 4> BisectSkipNumbers;
 };
 
 /// This class implements a mechanism to disable passes and individual
diff --git a/llvm/lib/IR/OptBisect.cpp b/llvm/lib/IR/OptBisect.cpp
index 29ca268408265..defb8a98f9716 100644
--- a/llvm/lib/IR/OptBisect.cpp
+++ b/llvm/lib/IR/OptBisect.cpp
@@ -37,6 +37,11 @@ static cl::opt<int> OptBisectLimit("opt-bisect-limit", cl::Hidden,
                                    }),
                                    cl::desc("Maximum optimization to perform"));
 
+static cl::opt<int> OptBisectSkip(
+    "opt-bisect-skip", cl::Hidden, cl::init(OptBisect::Disabled), cl::Optional,
+    cl::cb<void, int>([](int PassNum) { getOptBisector().addSkip(PassNum); }),
+    cl::desc("Skip pass at the given index in the optimization pipeline"));
+
 static cl::opt<bool> OptBisectVerbose(
     "opt-bisect-verbose",
     cl::desc("Show verbose output when opt-bisect-limit is set"), cl::Hidden,
@@ -66,7 +71,9 @@ bool OptBisect::shouldRunPass(StringRef PassName,
   assert(isEnabled());
 
   int CurBisectNum = ++LastBisectNum;
-  bool ShouldRun = (BisectLimit == -1 || CurBisectNum <= BisectLimit);
+  bool ShouldRun = (BisectLimit == -1 || BisectLimit == Disabled ||
+                    CurBisectNum <= BisectLimit) &&
+                   !BisectSkipNumbers.contains(CurBisectNum);
   if (OptBisectVerbose)
     printPassMessage(PassName, CurBisectNum, IRDescription, ShouldRun);
   return ShouldRun;
diff --git a/llvm/test/Other/opt-bisect-skip.ll b/llvm/test/Other/opt-bisect-skip.ll
new file mode 100644
index 0000000000000..da94ad6230527
--- /dev/null
+++ b/llvm/test/Other/opt-bisect-skip.ll
@@ -0,0 +1,15 @@
+; Test that verifies functionality for -opt-bisect-skip
+
+; RUN: opt -O1 -opt-bisect-skip=3 -opt-bisect-skip=7 %s 2>&1 | FileCheck %s --check-prefix=CHECK-DISABLE-PASS
+; CHECK-DISABLE-PASS: BISECT: running pass (1) annotation2metadata on [module]
+; CHECK-DISABLE-PASS: BISECT: running pass (2) forceattrs on [module]
+; CHECK-DISABLE-PASS: BISECT: NOT running pass (3) inferattrs on [module]
+; CHECK-DISABLE-PASS: BISECT: running pass (4) lower-expect on foo
+; CHECK-DISABLE-PASS: BISECT: running pass (5) simplifycfg on foo
+; CHECK-DISABLE-PASS: BISECT: running pass (6) sroa on foo
+; CHECK-DISABLE-PASS: BISECT: NOT running pass (7) early-cse on foo
+; CHECK-DISABLE-PASS: BISECT: running pass (8) openmp-opt on [module]
+
+define void @foo() {
+  ret void
+}
\ No newline at end of file

AlexMaclean

Nice, I personally use this feature in our down-stream fork and find it to be a nice complement to --opt-bisect-limit when investigating correctness issues. Please wait for some more review prior to landing.

llvm/test/Other/opt-bisect-skip.ll

llvm/lib/IR/OptBisect.cpp

github-actions · 2025-08-20T22:02:00Z

✅ With the latest revision this PR passed the C/C++ code formatter.

llvm/include/llvm/IR/OptBisect.h

nikic

I think it would be better to make opt-bisect work the same way that debug-counter works nowadays, that is by specifying a list of ranges. So that you can use something like -opt-bisect=1-1000 to get something similar to the current opt-bisect-limit, but also -opt-bisect=1-10,20-30 to skip a certain range of passes, etc.

YonahGoldberg · 2025-09-04T17:56:18Z

@Artem-B @nikic when you have a chance please look over the newest version of this. Here are the updates:

With OptBisect you can now do -opt-bisect=1-10,20-30 like Nikita asked. -opt-bisect-limit=N just translates to -opt-bisect=1-N
Since OptBisect and DebugCounter now have the same range based command line semantics, I refactored the support for range parsing into a separate utility in LLVMSupport.
DebugCounter supported ranges, but there was no lit-test to test this functionality, so I added one.

nikic · 2025-09-05T14:51:25Z

llvm/include/llvm/Support/Range.h

+  int64_t Begin;
+  int64_t End;
+
+  Range(const int64_t Begin, const int64_t End) : Begin(Begin), End(End) {}


Suggested change

Range(const int64_t Begin, const int64_t End) : Begin(Begin), End(End) {}

Range(int64_t Begin, int64_t End) : Begin(Begin), End(End) {}

Here and elsewhere, drop the const on non-reference/pointer params.

nikic · 2025-09-05T14:51:59Z

llvm/include/llvm/Support/Range.h

+  }
+
+  /// Get the size of this range
+  int64_t size() const { return End - Begin + 1; }


Should return an unsigned type?

True. I also realized that End - Begin can overflow. I don't know if what I implemented is overkill, but I made it safer so that it won't overflow.

nikic · 2025-09-05T14:52:50Z

llvm/include/llvm/Support/Range.h

+  }
+};
+
+/// Utility class for parsing and managing range specifications


Suggested change

/// Utility class for parsing and managing range specifications

/// Utility class for parsing and managing range specifications.

Here an elsewhere, end sentences with a period, including in doc comments.

nikic · 2025-09-05T14:53:57Z

llvm/include/llvm/Support/Range.h

+                                         const char Separator = ',');
+
+  /// Legacy interface for backward compatibility.
+  /// \deprecated Use the Expected<RangeList> version instead


This is a brand-new API, why have a deprecated interface?

nikic · 2025-09-05T14:56:15Z

llvm/lib/Support/Range.cpp

+
+  // Split by the specified separator
+  SmallVector<StringRef, 8> Parts;
+  Str.split(Parts, Separator, -1, false);


There is an iterator variant of this, no need to collect into a vector.

nikic · 2025-09-05T14:57:28Z

llvm/lib/Support/Range.cpp

+    Ranges.push_back(Range(Begin, End));
+  }
+
+  return std::move(Ranges);


Shouldn't need std::move here due to NRVO?

nikic · 2025-09-05T14:58:40Z

llvm/lib/Support/Range.cpp

+}
+
+std::string RangeUtils::rangesToString(const ArrayRef<Range> Ranges,
+                                       const char Separator) {


Is the to string functionality used for anything but testing? If not, we should only support the raw_ostream interface.

nikic · 2025-09-05T15:00:45Z

llvm/lib/IR/OptBisect.cpp

+      } else if (Limit > 0) {
+        // Convert limit to range 1-Limit
+        std::string RangeStr = Limit == 1 ? "1" : "1-" + llvm::utostr(Limit);
+        auto Ranges = RangeUtils::parseRanges(RangeStr);


Going through a string and then parsing it here is redundant? You can directly create the range.

nikic · 2025-09-05T15:02:37Z

llvm/test/Other/opt-bisect-ranges.ll

+; Test that verifies functionality for -opt-bisect with range specifications
+
+; Test basic range functionality: run passes 1-3 and 7-8
+; RUN: opt -O1 -opt-bisect=1-3,7-8 %s 2>&1 | FileCheck %s --check-prefix=CHECK-RANGES


It would be better to replace -O1 here with an explicit -passes list. This will save some poor person from updating this test when the O1 optimization pipeline changes.

YonahGoldberg · 2025-09-05T21:47:49Z

Hey @nikic I think I addressed all your comments, thanks for the feedback :)

The most major change is I decided to make Range into a class to preserve the Begin <= End invariant.

YonahGoldberg added 3 commits August 6, 2025 21:28

opt bisect skip

329098d

format

723514d

comment

431c13b

llvmbot added the llvm:ir label Aug 6, 2025

newline

ec6842e

AlexMaclean approved these changes Aug 19, 2025

View reviewed changes

llvm/test/Other/opt-bisect-skip.ll Outdated Show resolved Hide resolved

llvm/lib/IR/OptBisect.cpp Outdated Show resolved Hide resolved

AlexMaclean requested review from nikic, snehasish and mtrofin August 19, 2025 15:33

renaming

6b3ed99

format

895a1bb

Artem-B reviewed Aug 27, 2025

View reviewed changes

llvm/include/llvm/IR/OptBisect.h Outdated Show resolved Hide resolved

nikic reviewed Aug 27, 2025

View reviewed changes

YonahGoldberg added 3 commits September 3, 2025 04:33

range support

4b38209

small changes

d3f5add

remove opt-disable-indices

6f20867

llvmbot added the llvm:support label Sep 3, 2025

YonahGoldberg added 10 commits September 3, 2025 21:07

remove

5f87193

cleanup

8ab62b3

format

c3e5c7d

use Expect for Range

4cf5a99

format

0eefd68

fix hang

a159538

fix error handling tests

3215304

format

af7873f

cleanup

c27ce56

clang format

6c4fea1

fix minor formatting bug

5e1d443

YonahGoldberg changed the title ~~[OptBisect] Add opt-bisect-skip option for skipping passes during bisection~~ [OptBisect] Add support for selecting ranges of passes and refactor DebugCounter to use a shared Range API. Sep 4, 2025

YonahGoldberg added 5 commits September 4, 2025 18:56

small bug

466991d

format

0eaab13

allow -opt-bisect=0 and -opt-bisect=-1

5ce4228

comment change

095c358

format

dce302a

nikic reviewed Sep 5, 2025

View reviewed changes

YonahGoldberg added 11 commits September 5, 2025 20:18

cleanup

2b779d3

format

5a80bfb

more cleanup

3616aa6

format

5882584

range test cleanup

0f0499c

format

1615205

move range from struct -> class

98b2256

better range size function

debce85

range size checks

7db719f

format

7f44558

format

6924b7b

	Range(const int64_t Begin, const int64_t End) : Begin(Begin), End(End) {}
	Range(int64_t Begin, int64_t End) : Begin(Begin), End(End) {}

	/// Utility class for parsing and managing range specifications
	/// Utility class for parsing and managing range specifications.

[OptBisect] Add support for selecting ranges of passes and refactor DebugCounter to use a shared Range API. #152393

Are you sure you want to change the base?

[OptBisect] Add support for selecting ranges of passes and refactor DebugCounter to use a shared Range API. #152393

Conversation

YonahGoldberg commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

YonahGoldberg commented Aug 6, 2025

Uh oh!

llvmbot commented Aug 6, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

AlexMaclean left a comment

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Uh oh!

github-actions bot commented Aug 20, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Uh oh!

Uh oh!

nikic left a comment

Choose a reason for hiding this comment

Uh oh!

YonahGoldberg commented Sep 4, 2025

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

YonahGoldberg commented Sep 5, 2025

Uh oh!

Uh oh!

YonahGoldberg commented Aug 6, 2025 •

edited

Loading

llvmbot commented Aug 6, 2025 •

edited

Loading

github-actions bot commented Aug 20, 2025 •

edited

Loading